Fairness Index Measures to Evaluate Bias in Biometric Recognition

The demographic disparity of biometric systems has led to serious concerns regarding their societal impact as well applicability such in private and public domains. A quantitative evaluation fairness is an important step towards understanding, assessment, mitigation bias applications. While few, existing measures are based on post-decision data (such verification accuracy) systems, we discuss how pre-decision (score distributions) provide useful insights fairness. In this paper, introduce multiple measures, the statistical characteristics score distributions, for a generic system. We also propose different variants each measure depending contribution from constituent groups needs be combined final measure. case, behavior been illustrated numerically graphically synthetic data. imbalance benchmarking datasets often overlooked during assessment. novel weighing strategy reduce effect through non-linear function sample sizes groups. proposed independent modality, thus, applicable across commonly used modalities (e.g., face, fingerprint, etc.).